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Objectives:  Tracking  vaccine  reactions  and  adverse  events  during  a  large-scale  vaccination  program 
such  as  the  recent  smallpox  program  or  a  pandemic  flu  outbreak  will  be  a  challenge.  We  report  on 
vaccine  reaction  data  collected  using  a  novel  telephone-  and  web-based  electronic  reporting  system. 
The  system  was  used  to  monitor  vaccinees  during  the  U.S.  Army’s  smallpox  vaccination  campaign, 
which  was  part  of  the  national  program  to  prepare  against  biological  attack.  In  addition,  we  report 
on  the  time  course  of  events  after  smallpox  vaccination  based  on  the  self-reported  data  and  evaluate 
the  validity  and  reliability  of  self-reported  take  information  after  smallpox  vaccination. 

Methods:  A  prospective  cohort  of  subjects  receiving  the  smallpox  vaccination  volunteered  to  use 
an  electronic  monitoring  system  to  track  and  report  their  vaccination  reactions. 

Results:  Users  made  6.8  ±  6.2  (mean  ±  SD)  reports  using  the  electronic  monitoring  system.  The 
sensitivity  and  positive  predictive  value  of  self-reported  takes  were  high,  98.8%  and  99.6%,  respec¬ 
tively.  The  vaccination-site  reactions  progressed  faster  for  revaccinees  than  first-time  vaccinees. 

Conclusions:  Simplc-to-use  telephone/Internet-based  technology  allowed  detailed  self-recording  of 
response  to  smallpox  vaccination  among  outpatients.  Self-reports  on  site  appearance  were  sufficient 
to  determine  vaccine  takes  in  most  vaccinees.  During  a  mass  vaccination  event,  an  electronic  moni¬ 
toring  system  could  facilitate  tracking  of  vaccine  reactions,  including  providing  an  early  warning 
system  for  adverse  events,  and  might  reduce  the  burden  associated  with  follow-up  visits  with  health¬ 
care  professionals. 


The  Department  of  Health  and  Human  Services 
(DHHS)  and  the  Centers  for  Disease  Control  and 
Prevention  (CDC)  have  emphasized  the  importance  of 
public  health  planning  for  bioterrorism  events  such  as  a 
smallpox  attack  and  emerging  infectious  diseases  such  as 
pandemic  flu.  CDC  cooperative  agreements  have  re¬ 
quired  state  public  health  departments  to  develop  small¬ 
pox  vaccination  plans,  and  stales  are  now  being  asked  to 


develop  plans  for  an  influenza  pandemic.  In  addition  to 
the  usual  surveillance  functions  of  health  departments, 
these  agencies  may  need  to  set  up  mass  vaccination  clin¬ 
ics,  handle  vaccine  or  prophylactic  drugs,  determine  who 
should  get  vaccinated,  and  monitor  clinical  reactions  to 
the  vaccines.1  However,  few  methods  exist  for  real-time 
tracking  of  vaccinee  experiences.  The  Vaccine  Adverse 
Events  Reporting  System  (VAERS)  collects  spontaneous 
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reports  of  adverse  events  after  vaccination,  but  VAERS 
does  not  actively  solicit  participants  before  vaccination, 
nor  does  it  inform  clinicians  of  the  adverse  events  of  their 
patients  promptly  after  patients  experience  the  event. 
Furthermore,  it  does  not  capture  information  on  normal 
vaccine  reactions. 

In  addition,  data  on  the  natural  history  of  smallpox 
vaccination  arc  dated,  as  the  U.S.  has  not  routinely  vacci¬ 
nated  civilians  against  smallpox  since  1972.  Influenza 
vaccine  reactions  are  better  defined,  although  the  imple¬ 
mentation  difficulties  of  the  swine  flu  program  of  1976 
should  remind  planners  of  the  importance  of  good  moni¬ 
toring.  DHHS  recently  posted  draft  guidelines  on  prepar¬ 
ing  for  and  handling  a  pandemic  influenza  outbreak.2 
While  vaccination  for  influenza  is  not  as  complicated  as 
smallpox  vaccination,  the  plan  recognizes  that  adverse 
events  will  need  to  be  tracked  and  reported  in  the  case  of 
an  influenza  pandemic.2  It  is  not  clear  if  VAERS  will  be 
sufficient  for  this  purpose. 

Recent  advances  in  telephone  and  web-based  data  col¬ 
lection  can  streamline  the  process  of  monitoring  vaccina¬ 
tion  reactions.  Self-report  of  vaccination-site  appearance 
and  symptoms,  if  clinically  accurate,  could  potentially 
provide  an  early  warning  of  adverse  reactions  and,  partic¬ 
ularly  for  smallpox,  obviate  the  need  for  paper  diaries 
and  the  need  for  a  postvaccination  check  with  a  health¬ 
care  provider  and  its  attendant  costs.  We  tested  such  a 
monitoring  system  during  2003  in  the  Department  of 
Defense  (DoD)  smallpox  vaccination  program,  replacing 
the  CDC  recommended1  written  diary  of  symptoms.  Here 
we  report  the  frequency  of  signs  and  symptoms  reported 
by  vaccinees  and  the  sensitivity  and  specificity  of  using 
self-reports  to  determine  successful  vaccination.  We  also 
report  on  the  natural  progression  of  vaccine  reaction  in 
both  naive  vaccinees  and  previously  vaccinated  individu¬ 
als,  because  detailed  objective  data  have  not  been  avail¬ 
able  previously. 


METHODS 

Subjects 

Institutional  Review  Board  (IRB)  approval  for  this  re¬ 
search  was  obtained  from  the  RAND  Human  Subjects 
Protection  Committee,  from  the  Abt  Associates  IRB.  and 
from  the  IRBs  at  Brooke  Army  Medical  Center  and  Wal¬ 
ter  Reed  Army  Medical  Center. 

Military  personnel  and  civilian  DoD  employees  slated 
to  receive  smallpox  vaccine  as  part  of  the  DoD  vaccina¬ 
tion  program  were  invited  to  participate  in  the  study.  The 
study  was  conducted  at  four  sites:  Damall  Army  Com¬ 
munity  Hospital,  Fort  Hood,  Texas;  Womack  Army 
Medical  Center.  Fort  Bragg,  North  Carolina:  DeWitt 


Army  Community  Hospital,  Fort  Belvoir,  Virginia;  and 
the  DiLorenzo  Tricare  Health  Clinic  at  the  Pentagon.  Ar¬ 
lington.  Virginia.  Reports  were  received  from  March  to 
September  2003. 

Medical  screening  of  the  vaccinees  and  vaccination 
procedures  are  described  elsewhere.3  All  vaccinees  were 
asked  to  return  to  the  clinic  to  have  their  vaccine  sites 
checked  6-8  days  after  receiving  the  vaccination.  At  the 
point  of  vaccination,  vaccine  recipients  were  offered  the 
opportunity  to  report  vaccination  signs  and  symptoms  via 
an  automated  system  developed  by  Voxiva  Corporation 
(Washington,  DC).  Participation  in  the  electronic  moni¬ 
toring  system  was  entirely  voluntary.  Participants  regis¬ 
tered  for  the  system  and  received  instructions  on  its  use. 
including  a  pocket-sized  color  brochure  depicting  ex¬ 
pected  vaccination-site  responses.  Beginning  on  the  day 
of  vaccination  and  continuing  for  4  weeks,  vaccine  recip¬ 
ients  were  asked  to  call  the  telephone-based  system  or 
log  in  to  the  Internet  web-based  system  to  report  the  vac¬ 
cination-site  response  and  whether  any  signs  or  symp¬ 
toms  had  occurred.  To  promote  compliance,  a  call  center 
was  used  to  follow  up  with  nonrespondents.  At  the  start 
of  the  study,  trained  interviewers  from  the  call  center 
called  vaccinees  if  they  missed  any  daily  report.  Midway 
through  the  study,  DoD  requested  a  procedure  change, 
and  call  frequency  was  reduced  to  calling  vaccinees  who 
had  not  reported  on  day  3,  6.  or  9  postvaccination. 

The  automated,  password-protected  telephone  system 
provided  an  audio  introduction  and  then  asked  vaccinees 
to  enter  2-digit  codes  for  specific  signs  or  symptoms.  Al¬ 
ternately,  vaccinees  could  log  onto  an  Internet  website  to 
record  corresponding  data.  All  vaccinees  had  both  report¬ 
ing  options.  Participants  who  spoke  with  the  call  center 
could  make  reports  directly  to  the  call  center  staff. 

Each  vaccinee  had  the  option  to  request  contact  with  a 
nurse,  using  either  the  call  center  or  the  automated  sys¬ 
tem;  the  nurse  answered  the  vaccinee's  questions  and 
recommended  referral  to  additional  medical  care  if 
needed.  Data  from  vaccinees  whose  reports  contained 
triggers  (e.g.,  reports  of  chest  pain  or  extreme  concern 
about  their  vaccine  reaction)  were  automatically  sent — 
by  email.  Short  Message  Service,  or  both — to  the  health¬ 
care  professional  managing  the  vaccination  process  for 
that  clinic,  who  then  had  responsibility  for  arranging  fol¬ 
low-up  for  the  vaccinee. 

The  following  fixed-choice  vaccination-site  descrip¬ 
tions  were  used  for  vaccinees  to  report  site  reactions: 
none,  colored  spot  (to  identify  macules),  bump  (to  iden¬ 
tify  papules),  reddish  blister  (to  identify  vesicles), 
whitish  blister  (to  identify  pustules),  scab,  scab  fell  off, 
unknown,  and  did  not  look.  Vaccination-site  symptoms 
included:  itching,  leaking  fluid  (to  identify  exudate), 
pain,  local  rash,  streaking,  swelling,  warmth,  and  ban¬ 
dage  reaction.  Fixed  choices  for  systemic  signs  or  symp- 
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loms  included:  chest  pain,  chills,  eye  infection,  feeling 
lousy,  fever,  headache,  joint  ache,  muscle  ache,  rash 
(general),  swollen  lymph  nodes,  and  other.  Respondents 
could  dictate  or  type  in  symptoms  that  were  not  listed. 

Data 

When  enrolling  in  the  system,  vaccinees  provided  in¬ 
formation  about  their  age,  gender,  race,  military  rank 
(E1-E9,  W1-W5,  01-09.  or  Civilian),  and  outpatient 
clinic.  The  clinic  provided  vaccination-response  status 
and  vaccinalion/revaccination  status  based  on  medical 
records.  An  analytic  data  set  was  created  by  merging  the 
electronic  monitoring  system  data  set,  including  demo¬ 
graphics  and  daily  reports,  with  data  from  the  DoD  vacci¬ 
nation  program. 

Electronic  reports  were  analyzed  for  site  progression 
and  compared  with  the  health  professional's  assessment 
of  vaccination  response.  The  CDC  definition  of  a  take  or 
“major  reaction"  is  "a  pustular  lesion  or  an  area  of  defi¬ 
nite  induration  or  congestion  surrounding  a  central  le¬ 
sion.  which  can  be  a  scab  or  an  ulcer.”1  To  maintain  con¬ 
sistency  with  that  definition,  a  “bump,”  “reddish  blister,” 
or  “whitish  blister”  had  to  be  reported  in  the  electronic 
self-reports  to  be  considered  a  take  for  purposes  of  this 
analysis.  To  be  considered  a  nonrespondcr  (nontake),  a 
vaccinee  had  to  make  a  report  of  "none”  or  “colored 
spot"  at  least  once  on  or  after  day  6  after  vaccination  (the 
first  day  recommended  by  the  World  Health  Organiza¬ 
tion  [WHO]  for  take  checks)  without  reporting  further 
progression  to  bump,  blister,  or  scab. 

Statistical  Methods 

We  used  Chi-square  tests  and  two-sample  /-tests  to 
compare  demographic  characteristics  across  different 


population  groups.  To  compare  the  incidence  of  specific 
symptom  reporting  between  first-time  vaccinees  and  re- 
vaccinees,  logistic  regression  models  were  used.  Nega¬ 
tive  binomial  regression  models  were  used  to  compare 
revaccinees  and  first-time  vaccinees  regarding  total  num¬ 
ber  of  distinct  symptoms.  Similarly,  for  total  number  of 
symptoms  per  report,  due  to  the  skewed  distribution  of 
the  reports,  categories  were  formed  and  multinomial  re¬ 
gression  models  were  used.  To  compare  timing  of  vac¬ 
cine-site  appearances  and  selected  symptoms  between  re¬ 
vaccinees  and  first-time  vaccinees.  time-to-event  models, 
specifically  the  Cox  proportional  hazards  model,  were 
used.  All  of  these  regression  models  included  gender, 
race,  outpatient  clinic,  military  rank,  and  revaccination 
status  as  independent  variables.  Age  could  not  be  in¬ 
cluded  in  these  models  due  to  high  collinearity  with  re- 
vaccination  status.  All  statistical  procedures  were  imple¬ 
mented  using  SAS  (SAS  System  version  8.2,  Cary,  NC). 

RESULTS 

Use  of  the  System 

A  total  of  1 ,649  vaccinees  volunteered  to  use  the  elec¬ 
tronic  monitoring  system.  Most  were  from  Fort  Hood 
(n  =  715)  or  the  Pentagon  (n  =  822).  Unfortunately,  be¬ 
cause  the  system  was  implemented  during  the  prepara¬ 
tion  for  the  war  in  Iraq,  the  number  of  people  who  were 
approached  and  declined  to  participate  was  not  collected. 
Most  of  those  who  declined  stated  that  they  were  deploy¬ 
ing  within  a  week  and  would  not  be  able  to  fully  partici¬ 
pate.  Demographic  characteristics  of  volunteers  from 
Forts  Belvoir,  Bragg,  and  Hood  were  similar  and  are  re¬ 
ported  in  aggregate  as  “Army  posts”  (Table  1).  Volun¬ 
teers  from  the  Pentagon  were  more  likely  to  be  officers. 


Table  1.  Demographics  of  Vaccinees  by  Clinic 


Army  posts 

Pentagon 

Number  who  signed  up  for  system 

827 

822 

Number  who  used  the  system 

546  (66%) 

708  (86%) 

Mean  number  of  reports  (±SD) 

4.4  ±  5.0 

8.6  r  6.3 

Median  number  of  reports 

2 

7 

Mean  age  (rSD) 

28  ±  8 

42  r  9 

%  Officer 

7.6 

65 

%  Enlisted/Warrant 

91 

20 

%  Civilian 

1.6 

15 

9c  White" 

55 

81 

%  Black" 

26 

11 

%  Male 

93 

87 

9c  Revaccinee 

30 

87 

'Race  was  categorized  as  white,  black,  or  other. 
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Caucasian,  revaccinees,  and  older.  Of  the  1,649  who  en¬ 
rolled  in  the  electronic  monitoring  system.  1,254  (76.0%) 
used  the  system  at  least  once,  making  a  total  of  8,510 
electronic  reports  (Figure  I).  The  mean  number  of  re¬ 
ports  made  per  user  was  6.8  ±  6.2  (mean  ±  SD),  and  the 
median  number  of  reports  was  5  (range  1-28). 

Take  Checks 

Of  the  1,254  respondents,  530  vaccinees  had  both 
enough  reports  to  interpret  a  take  and  had  their  health  pro¬ 
fessional  take  check  recorded  in  their  DoD  vaccination 
records  (Table  2).  The  electronic  take-check  data  have  a 
sensitivity  of  98.9%  (521/527),  specificity  of  33.3%  ( 1/3), 
positive  predictive  value  of  99.6%  (521/523),  and  nega¬ 
tive  predictive  value  of  14.3%  (1/7).  For  this  calculation, 
we  have  treated  the  DoD  vaccine  records  as  the  gold  stan¬ 
dard.  Making  the  requirements  for  a  take  more  restrictive 
(at  least  reporting  a  blister  or  reporting  a  bump  and  itch¬ 
ing)  did  not  change  these  values.  If  respondents  did  not 
make  enough  electronic  reports  to  adequately  determine 
their  take  status  (320  respondents),  their  data  were  consid¬ 
ered  “missing”  electronic  data.  Similarly,  in  499  cases. 


electronic  data  were  available,  but  a  take  determination 
was  not  recorded  in  the  DoD  clinic  vaccination  records, 
and  data  were  considered  “missing." 

The  DoD  take  records  and  the  interpretation  of  elec¬ 
tronic  reports  do  not  match  for  8  cases  for  whom  both  self- 
report  and  take-check  data  are  available.  Two  vaccinees 
whose  DoD  records  indicated  the  vaccine  did  not  take  had 
electronic  reports  that  met  the  definition  for  take.  In  one  of 
these  two  cases,  the  clinic’s  take  check  was  probably  per¬ 
formed  before  the  recommended  time  frame  of  6-8  days, 
in  that  on  day  4  the  vaccination  site  was  read  as  “no  take,” 
but  the  vaccinee  made  an  electronic  report  of  a  bump  on 
day  5  and  continued  to  report  through  the  scab  falling  off. 
The  other  vaccinee  also  reported  the  full  progression  of 
site  appearance,  from  bump  through  scab  falling  off,  and 
the  take  check  was  performed  on  day  10.  We  were  unable 
to  determine  why  the  health  professional  take  assessment 
for  this  person  does  not  match  the  electronic  records.  We 
suspect  a  data  entry  error,  because  the  date  of  vaccination 
in  the  military  health  record  does  not  match  the  date  of 
vaccination  in  the  electronic  reporting  system. 

Six  vaccinees  made  electronic  reports  that  we  defined 
as  nontakes  but  clinical  records  indicated  as  a  take.  These 


Figure  1 .  Reporting  Rates.  The  number  of  reports  made  by  vaccinees  is  graphed  as  a  percentage  of  vaccinees  who 
signed  up  for  the  electronic  monitoring  system.  The  mean  number  of  reports  made  per  user  was  6.8  ±  f>.2  (mean  ± 
SD),  and  the  median  number  of  reports  was  5;  73.4%  of  reports  were  made  on  the  web,  15.3%  were  made  by  calling 
into  the  system,  and  1 1.3%  were  made  directly  by  the  call  center. 
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Table  2.  Comparison  of  Clinic  Take-Check  Decisions  with  Electronic 
Descriptions  Empirically  Defined  as  a  Take 


Electronic  reports 


Take 

No  take 

Missing 

Totals 

DoD 

Take 

521 

6 

222 

749 

Vaccine 

No  take 

2 

1 

3 

6 

Records 

Missing 

393 

11 

95 

499 

Totals 

916 

18 

320 

1.254 

6  vaccinees,  4  revaccinees  and  2  first-time  vaccinees, 
made  an  average  of  4.5  electronic  reports.  None  of  the  6 
reported  anything  other  than  none  (no  reaction)  or  col¬ 
ored  spot,  and  all  reported  at  least  once  on  or  after  day  8, 
the  last  day  recommended  for  a  take  check. 

Progression  of  Vaccination  Site  and  Symptoms 

Electronic  records  of  vaccinees  who  had  a  take  and 
whose  vaccination/revaccination  status  was  known  were 
analyzed  for  progression  of  their  vaccination  site.  We 
used  only  the  first  report  of  a  specific  site-appearance  for 
each  vaccinee  to  calculate  this  progression.  A  total  of 
1,683  reports  from  245  first-time  vaccinees  and  6,191  re¬ 
ports  from  66 1  revaccinees  are  included  in  this  analysis 
(Figure  2).  The  average  day  of  first  report  of  the  red  blis¬ 
ter,  white  blister,  scab  formation  (all  p  <  0.001 )  and  scab 
separation  (p  =  0.02)  are  all  earlier  for  revaccinees  than 
for  first-time  vaccinees. 


The  length  of  time  that  a  scab  persisted  was  often 
longer  than  the  21  days  previously  reported.1  Of  the  293 
users  who  made  electronic  reports  21-28  days  after  get¬ 
ting  vaccinated,  170  (58%)  reported  a  scab  during  that 
time,  including  30  (of  72)  who  reported  a  scab  on  day  28. 

Symptoms  were  reported  by  1,212  of  the  1,254  system 
users.  The  frequencies  of  symptom  reporting  are  summa¬ 
rized  in  Table  3.  Users  reported  3.8  ±  3.4  distinct  symp¬ 
toms  (1.6  ±  1.9  symptoms  per  electronic  report,  mean  ± 
SD).  First-time  vaccinees  generally  reported  more  dis¬ 
tinct  symptoms  (4.3  vs.  3.6)  and  reported  more  symp¬ 
toms  per  electronic  report  (2.2  vs.  1.3)  than  did  revacci¬ 
nees.  The  p  values  listed  in  Table  3  comparing  first-time 
vaccinees  and  revaccinees  are  from  regression  models 
that  adjust  for  gender,  race,  clinic  site,  and  rank.  The 
most  common  symptoms  reported  are  graphed  by  post¬ 
vaccination  day  in  Figure  3.  Itching  was  the  most  fre¬ 
quently  reported  symptom,  peaking  early  during  the  first 
week.  The  rest  of  the  symptoms  peaked  around  days  6 
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Figure  2.  Progression  of  Vaccine  Site  Appearance.  Data  are  the  mean  ( ±SD)  of  the  first  day  that  vaccinees  re¬ 
ported  each  site  appearance.  Results  of  lime-to-event  regression  models  indicate  revaccinees  report  site  progression 
earlier  compared  to  first-time  vaccinees:  colored  spot  (p  =  0.06),  bump  (]>  =  0.13),  red  blister,  white  blister,  scab  for¬ 
mation  (all  p  <  0.001 ).  scab  separation  (p  =  0.02). 


ELECTRONIC  MONITORING  SYSTEM  FOR  SELF-REPORTING  SMALLPOX 


203 


Table  3.  Comparison  or  Reported  Symptom  Data 

by  Vaccination  Status 

First-time  vaccinee 
(n  =  439) 

Revaccinee 
(n  =  773) 

Mean  (SD) 

Mean  (SD) 

p -value* 

All  symptoms  9.5(13  9) 

All  symptoms  (per  report)  2.2  (2.4) 

All  symptoms  (distinct)  4.3  (4  Q) 

9.6  (12.2) 

1.3  (1.4) 

3.6  (3.0) 

NA 

0.02 

<0.001 

Itching 
Leaking  fluid 
Pain 

Local  rash 
Streaking 
Swelling 
Warmth 

Bandage  reaction 
Incidence  of  other  symptoms 
Chest  pain 
Chills 

Eye  infection 
Feeling  lousy 
Fever 
Headache 
Joint  ache 
Muscle  ache 
Rash  (general) 

Swollen  lymph  nodes 
Other 


Percentage 

Percentage 

69.0 

84.7 

0.07 

43.5 

44.4 

<0.01 

32.3 

21.2 

<0.01 

17.3 

12.8 

0.01 

4.3 

2.1 

0.19 

27.8 

22.3 

0.01 

15.0 

13.1 

0.08 

34.4 

23.0 

<0.01 

5.2 

2.3 

0.46 

12.1 

6.1 

0.05 

0.7 

0.6 

0.6) 

32.8 

24.2 

0.01 

12.8 

5.8 

0.01 

29.8 

24.1 

<0.01 

18.2 

12.9 

0.07 

24.6 

21.2 

0.18 

4.6 

3.8 

0.34 

31.4 

16.0 

<0.001 

16.6 

16.0 

0.14 

•■Comparisons  are  from  regression  models  that  adjust  for  gender,  race,  clinic  site,  and  rank. 


and  7  for  revaccinees  and  days  8—10  for  first-time  vacci- 
nees.  As  was  the  case  with  site  progression,  the  average 
first  day  that  most  symptoms  were  reported  was  earlier 
for  revaccinees  than  first-time  vaccinees  (joint  ache, 
p  =  0.0 1 ;  itching,  local  rash,  warmth,  muscle  ache,  and 
swollen  lymph  nodes,  p  <  0.01;  pain,  leaking  fluid,  and 
swelling,  p  <  0.001). 

Of  the  1.254  users,  1 13  (9.0%)  made  at  least  one  report 
that  triggered  an  automatic  message  to  a  healthcare  pro¬ 
fessional  at  their  vaccination  clinic.  Forty-four  vaccinees 
(3.5%)  reported  chest  pain,  and  77  (6. 1  %)  reported  being 
extremely  concerned  about  their  vaccination.  The  authors 
do  not  have  follow-up  information  on  the  clinical  course 
of  these  patients,  but  they  were  included  in  other  surveil¬ 
lance  efforts.3 

DISCUSSION 

Advances  in  technology  have  the  capacity  to  revolu¬ 
tionize  both  clinical  care  and  public  health.  We  found 


that  recipients  of  smallpox  vaccine  were  able  to  accu¬ 
rately  assess  their  reaction  to  smallpox  vaccine  and  to 
report  it  using  telephone-  and  web-based  technology. 
We  believe  that  such  emerging  technologies  can  play  an 
important  role  in  the  consideration  of  future  vaccination 
campaigns.  Even  putting  bioterrorism  aside,  it  is  not  un¬ 
likely  that  a  novel  infectious  disease  or  pandemic  flu 
will  emerge  as  a  major  public  health  challenge  in  the  fu¬ 
ture  and  that  a  large  vaccination  campaign  could  be  re¬ 
quired. 

Specifically  with  regard  to  smallpox,  adverse  events 
from  the  recent  smallpox  vaccination  campaign  have 
been  documented  for  both  the  military  and  civilian 
populations.3  6  In  some  cases.  limited  symptom  data  also 
have  been  reported. u  However,  other  than  a  report  of  4.8 
vaccinees  with  potential  superinfcction,8  this  is  the  first 
detailed  report  of  the  progression  of  the  vaccination  site 
and  symptoms,  contrasting  first-time  vaccinees  and  re¬ 
vaccinees  in  this  contemporary  cohort.  The  vaccine  reac¬ 
tion  progressed  more  quickly  in  revaccinees  than  in  first- 
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Figure  3.  Reports  of  Symptoms  by  Postvaccination  Day  for  First-time  Vaccinees  (upper  graph)  and  Revac¬ 
cinees  (lower  graph).  Firsi-time  vaccinees  made  more  reports  of  symptoms  (both  frequency  of  reporting  and  num¬ 
ber  of  unique  symptoms  reported)  than  revaccinees.  Itching  was  the  most  commonly  reported  symptom  for  both 
groups,  peaking  during  the  first  week.  The  average  first  day  that  most  symptoms  were  reported  was  significantly  ear¬ 
lier  for  revaccinees  than  first-time  vaccinees  (see  text  for  p  values). 


lime  vaccinees,  while  first-time  vaccinees  were  more 
likely  to  report  symptoms  including  pain,  rash,  and 
swollen  lymph  nodes. 

We  found  that  overall  use  of  the  electronic  monitoring 
system  was  modest  but  that  the  sensitivity  and  positive 
predictive  value  of  self-report  of  a  vaccine  take  was  quite 


high.  If  the  reporting  frequency  could  be  improved,  clin¬ 
ics  could  rely  on  an  electronic  monitoring  system  to  mon¬ 
itor  vaccine  take  among  their  clients.  With  this  cohort, 
the  electronic  system  could  have  reduced  the  number  of 
return  take-check  visits  by  73%.  In  addition,  the  system 
is  designed  with  the  ability  to  automatically  notify  a 


ELECTRONIC  MONITORING  SYSTEM  FOR  SELF-REPORTING  SMALLPOX 


205 


healthcare  professional  when  a  vaccinee  reports  a  symp¬ 
tom  of  concern,  such  as  chest  pain. 

There  are  several  likely  explanations  for  the  relatively 
modest  use  of  the  system.  First,  the  system  began  opera¬ 
tion  at  the  height  of  deployment  in  preparation  for  the 
war  in  Iraq,  and  it  is  likely  that  both  the  salience  of  re¬ 
porting  and  access  to  telephones  and/or  the  Internet  were 
quite  limited.  In  addition,  unlike  in  a  public  health  emer¬ 
gency  where  people  would  likely  be  focused  on  the  emer¬ 
gency  itself,  vaccinees  here  were  focused  more  on  de¬ 
ployment  to  Iraq  and  the  impending  war  than  on  their 
vaccination.  However,  we  cannot  predict  whether  other 
distractions  will  inhibit  reporting.  Further,  this  was  a  pi¬ 
lot  study,  and  we  expect  that  the  recruitment  and  expla¬ 
nation  process  could  be  enhanced  in  ways  that  would 
increase  the  frequency  and  likelihood  of  reporting.  This 
first  test  of  the  electronic  system  did  not  include  auto¬ 
matic  email  reminders  or  incentives  for  reporting.  Addi¬ 
tional  use  of  electronic  reminders  and  modest  incentives 
(e.g.,  a  prepaid  phone  card)  may  increase  frequency  of 
reporting.  Use  of  a  call  center  was  important  in  stimulat¬ 
ing  reporting,  although  it  did  not  have  the  magnitude  of 
effect  we  had  hoped.  However,  based  on  the  results  of 
this  pilot,  we  are  able  to  identify  groups  at  highest  risk  of 
not  reporting  and  could  target  that  subpopulation  for  in¬ 
tensive  work  by  the  call  center.9 

These  results  also  reinforce  the  importance  of  perform¬ 
ing  the  take  check  after  sufficient  time  for  the  vaccination 
reaction  to  occur.  On  at  least  one  occasion,  a  vaccinee  re¬ 
turned  for  a  take  check  on  day  4  postvaccination,  before 
the  take  was  apparent  physically.  The  vaccination  site 
progressed  to  a  white  blister  by  day  6  and  would  proba¬ 
bly  have  been  correctly  interpreted  as  a  take  at  that  time. 
Furthermore,  the  results  suggest  that  the  ideal  time  for  a 
take  check  may  start  slightly  earlier  for  revaccinees  com¬ 
pared  with  first-time  vaccinees. 

Clinical  experience  passed  down  from  the  1960s  in¬ 
cluded  the  expectation  that  vaccinia  scabs  would  fall  off 
between  the  14th  and  21st  days.1  Our  observation  of 
longer  scab-retention  intervals  could  be  due  to  greater 
use  of  bandages  than  was  common  in  the  1960s.  lack  of 
an  evidence  basis  for  the  14-2 1  day  interval,  or  both. 

The  smallpox  vaccination  reaction  is  known  to  be  differ¬ 
ent  in  first-time  vaccinees  compared  with  revaccinees.10 
Our  site-appearance  and  symptom  results  confirm  this, 
with  each  stage  after  bump  and  each  of  the  site-specific 
symptoms  except  bandage  rash  and  streaking  occurring 
significantly  earlier  in  revaccinees  than  in  first-time  vacci¬ 
nees.  However,  age  and  vaccination  status  (first-time  or  re¬ 
vaccination)  are  collinear  variables,  so  we  were  unable  to 
adjust  for  age  in  our  models.  Potentially,  this  faster  matu¬ 
ration  is  due  to  the  activation  of  memory  cells  in  the  indi¬ 
vidual's  immune  system."  However,  other  biological  and 
nonbiological  explanations  may  account  for  the  faster  pro¬ 


gression  as  well.  In  this  study,  older  people,  who  are  the 
ones  more  likely  to  have  been  previously  vaccinated,  made 
more  reports  with  the  system.  The  potential  therefore  ex¬ 
ists  for  a  bias  based  on  the  frequency  of  reporting.  Older 
vaccinees  may  have  been  more  likely  to  report  the  first 
day  that  their  vaccination  site  changed  appearance,  which 
would  give  the  appearance  of  having  an  earlier  transition 
to  the  next  stage,  whereas  younger  vaccinees,  who  re¬ 
ported  less  frequently  per  capita,  may  not  have  reported  on 
the  first  day  that  the  vaccination  site  changed  appearance. 

Finally,  revaccinees  received  15  sticks  with  the  bifur¬ 
cated  needle,  while  first-time  vaccinees  received  3  sticks, 
per  the  FDA-licensed  dosing  guidelines.  We  do  not  know 
whether  15  sticks  in  first-time  vaccinees  would  have  in¬ 
duced  a  faster  immune  response,  either  because  a  larger 
dose  of  vaccine  was  delivered  or  because  more  local 
trauma  caused  a  greater  inflammatory  response,  which  may 
have  caused  a  faster  progression  of  the  vaccination  sites. 

This  electronic  monitoring  system  was  successful  in 
gathering  data  from  smallpox  vaccine  recipients.  Al¬ 
though  usage  was  modest,  the  data  suggest  that  even 
modest  usage  during  a  mass  vaccination  campaign  of  the 
general  public  could  facilitate  tracking  vaccine  reactions, 
including  providing  an  early  warning  system  for  adverse 
events  and  reducing  the  time  associated  with  follow-up 
visits  with  healthcare  professionals.  For  smallpox  vacci¬ 
nees,  reporting  just  2-3  times  during  the  first  10  days 
postvaccination  would  provide  enough  information  to 
demonstrate  successful  vaccination  in  most  cases. 
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